Data-Dependent Path Normalization in Neural Networks
نویسندگان
چکیده
We propose a unified framework for neural net normalization, regularization and optimization, which includes Path-SGD and Batch-Normalization and interpolates between them across two different dimensions. Through this framework we investigate issue of invariance of the optimization, data dependence and the connection with natural gradients.
منابع مشابه
Efficient Short-Term Electricity Load Forecasting Using Recurrent Neural Networks
Short term load forecasting (STLF) plays an important role in the economic and reliable operation ofpower systems. Electric load demand has a complex profile with many multivariable and nonlineardependencies. In this study, recurrent neural network (RNN) architecture is presented for STLF. Theproposed model is capable of forecasting next 24-hour load profile. The main feature in this networkis ...
متن کاملUse of Artificial Neural Networks and PCA to Predict Results of Infertility Treatment in the ICSI Method
Background: Intracytoplasmic sperm injection (ICSI) or microinjection is one of the most commonly used assisted reproductive technologies (ART) in the treatment of patients with infertility problems. At each stage of this treatment cycle, many dependent and independent variables may affect the results, according to which, estimating the accuracy of fertility rate for physicians will be difficul...
متن کاملDesign of an Intelligent Controller for Station Keeping, Attitude Control, and Path Tracking of a Quadrotor Using Recursive Neural Networks
During recent years there has been growing interest in unmanned aerial vehicles (UAVs). Moreover, the necessity to control and navigate these vehicles has attracted much attention from researchers in this field. This is mostly due to the fact that the interactions between turbulent airflows apply complex aerodynamic forces to the system. Since the dynamics of a quadrotor are non-linear and the ...
متن کاملPrediction-Based Portfolio Optimization Model for Iran’s Oil Dependent Stocks Using Data Mining Methods
This study applied a prediction-based portfolio optimization model to explore the results of portfolio predicament in the Tehran Stock Exchange. To this aim, first, the data mining approach was used to predict the petroleum products and chemical industry using clustering stock market data. Then, some effective factors, such as crude oil price, exchange rate, global interest rate, gold price, an...
متن کاملLayer Normalization
Training state-of-the-art, deep neural networks is computationally expensive. One way to reduce the training time is to normalize the activities of the neurons. A recently introduced technique called batch normalization uses the distribution of the summed input to a neuron over a mini-batch of training cases to compute a mean and variance which are then used to normalize the summed input to tha...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1511.06747 شماره
صفحات -
تاریخ انتشار 2015